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Abstract 

The phenomenal growth in the healthcare data has inspired us in investigating robust and scalable models for data mining. 
For classification problems Information Gain(IG) based Decision Tree is one of the popular choices. However, depending upon 
the nature of the dataset, IG based Decision Tree may not always perform well as it prefers the attribute with more number of 
distinct values as the splitting attribute. Healthcare datasets generally have many attributes and each attribute generally has many 
distinct values. In this paper, we have tried to focus on this characteristics of the datasets while analysing the performance of our 
proposed approach which is a variant of Decision Tree model and uses the concept of Correlation Ratio(CR). Unlike IG based 
approach, this CR based approach has no biasness towards the attribute with more number of distinct values. We have applied 
our model on some benchmark healthcare datasets to show the effectiveness of the proposed technique. 

Index Terms 

Data Mining, Healthcare, Decision Tree, Information Gain, Correlation Ratio. 


I. Introduction 

Due to the growth of Internet technology and healthcare software, data are available in abundance in unstructured form as all 
as structured form over the Internet. We are rich in information but lack of knowledge. So this led to the path to healthcare 
data mining. Healthcare data mining techniques find hidden but useful patterns from healthcare datasets. 

Diseases like heart-diseases, hepatitis, diabetes are very common among all ages of patients. Some alarming statistics regarding 
those diseases are given below: 

• One-third of all deaths in India will be caused by cardiovascular disease by the year 2020. ifTl 

• In middle-income countries, diabetes is one among the top 10 life threatening diseases. El. 

All these observations are made after some careful analysis of different healthcare datasets. And such observations often change 
the focus of the government policies. 

There are different data mining algorithms - such as Decision Tree algorithm 13], Naive Bayes Classifier lUlSllll, Neural 
Network model 0131, k-Nearest Neighbour algorithm^. Support Vector Machine QS, K-means 00, Bisecting K-means 
algorithm 0, Association Rule mining algorithms 1110 etc. Among them some are used for classification, some for clustering 
purposes and some others for finding easily interpretable rules for taking proper decision. Decision Tree algorithm is a very 
popular algorithm for classifications. It generally uses Information Gain (IG) as the criterion for splitting on an attribute. The 
attribute with the highest IG is chosen as the splitting attribute at each level. But Information Gain has some disadvantages 
like it prefers the attribute which has large number of distinct values. So if there is an attribute in the dataset like product-ID, 
then Information Gain approach will prefer to split on product-ID as because this attribute can uniquely identify each tuple 
in the set and this would result in a large number of partitions (as many as there are values), each one will have just one 
tuple. Since there will be no records with different class labels in each partition, the required information to classify data set D 
based on this partitioning based on Information Gain 0 principle would be Info product-ID(D)=0. Therefore, the information 
gained by partitioning on this attribute is highest. Therefore, such a partitioning is useless for classification 0. So it will not 
generalize the model. So, IG based approach is not effective for all types of datasets. Eor instance, if a dataset has different 
attributes with different numbers of distinct values, it prefers the attributes with more numbers of distinct values as splitting 
attributes though some other attributes with less number of distinct values may be more significant for classification. 

Due to this reason for some dataset IG based approach does not provide adequate accuracy. To overcome this we have proposed 
an approach which uses the concept of Correlation Ratio im or CR as the splitting criterion. This method has no such biasness. 


It considers that attribute for classification which is significant enough to identify at least one outcome class. The general CR 
method is suitable for quantative data. But our proposed CR based approach is applicable to nominal or categorical attributes 
also. 


The organization of the paper is as follows: Section |II] describes the related work done in this area. Section |I^ illustrates our 
proposed approach. The result and analysis of the proposed approach is shown in Section IV Section [V] concludes the paper 
and gives future direction of the work. 


II. Related Work 

Decision Tree technique has been popularly used for finding interesting patterns in health care datasets. We will discuss next 
some of the relevant works demonstrating this fact. 

Polat et al.lflU proposed a hybrid model for classification of multiclass dataset. For each class separate models based on C4.5 
Decision Tree algorithm has been constructed and the class for which the model is built is given positive class label and the 
rest of the classes are assigned negative class label. The proposed model showed significant performance improvement over 
the traditional C4.5 Decision Tree based model. As optimization of dataset can improve classification accuracy, some more 
methods like Homogeneity-based algorithm (HBA) etc have been proposed by Pham et al. ns. This algorithm was used in 
association with standard classification algorithms as SVM, DT and ANN to enhance their performances. The four parameters 
of HBA were then optimized by Genetic Algorithm. The proposed approach showed significant performance improvement over 
standard approaches. Decision Tree induction method has a wide variety of applications as discussed previously. Changala 
et al. OH have discussed different aspects of the Decision Tree induction method in the paper. Since most of the learning 
algorithms require the dataset to be in memory, it is a matter of concern for huge datasets. So, in this paper the scalability 
issues have also been discussed. 

Karaolis et al. 03 used Decision Tree based models to find out the risk factors for three types of Coronary heart disease events 
- myocardial infarction (MI), percutaneous coronary intervention (PCI), and coronary artery bypass graft surgery (CABG). The 
models for PCI and CABG have performed well with 75% classification accuracy compared to the model for MI. A predictive 
model for determining the ability of the persons affected with dementia to take help of a technology based on mobile phone 
based video streaming system has been developed by Zhang et al. M- The dataset was having two classes : Adopter and 
Non-Adopter. Popular classification algorithms were used for building models. Experimental results shows that among all 
the models DT, NN, SVM and kNN based models performed well. G. Sathyadevi ca proposed to build intelligent decision 
support system for Hepatitis disease diagnosis. At first most relevant attributes were selected based on some threshold value or 
some condition. Then CART(Classification And Regression Trees) Decision Tree algorithm was applied on the dataset which 
showed 83.2% accuracy and it was relatively higher as compared to the accuracy obtained from models using ID3 and C4.5 
Decision Tree algorithms. Some significant rules were extracted after constructing the Decision Tree using CART. Decision 
Tree as a prediction model has also been used by the author in ifTSll to predict hepatitis C virus(HCV) polyprotein cleavage 
sites. The challenge was to collect accurate data. The model gave a very good result with a 96% accuracy which was slightly 
lesser than the accuracy(97%) of the model based on Support Vector Machine(SVM). 

Overall the above survey demonstrates the effectiveness of Decision Tree technique for efficient healthcare. 

III. Proposed Approach 

In this section, we give the details of our proposed Correlation Ratio based Decision Tree construction approach. 

A. Overview of Correlation Ratio 

Sometimes, the expected outcome of our learning algorithm is some categorical values like “yes/no”. The Correlation Coefficient 
method is suitable for applications where the outcome is quantitative, thus it cannot be applied in those cases where categorical 
outcome is desired. To sort out this problem, the CR lfTTl method can be applied. 

The CR method can be employed to partition the sample dataset into different categories according to the observed outcome. 
A significant attribute is one which can identify at least one outcome class where the average value of the attribute and the 
average on all classes are remarkably different, otherwise that attribute would not be useful to identify any outcome. Table [I] 
provides a summary of the different notations used in our proposed approach and the corresponding meanings. 

Suppose that there is a set of £ tuples in a dataset. Let the number of times that outcome y G Y occurs is iy, so that the 
dataset can be partitioned by their outcome as follows: 



TABLE I: Some frequently used terms 


Notation 

Corresponding Meaning 

Y 

Set of outcomes or class labels 

1 

Total number of samples 

ly 

Number of times that outcome y QY occurs is ly 

^jy 

i-th attribute value of the j-th tuple among the £y 
samples having outcome y 


Average of the i-th attribute from all sample 
vectors within each outcome class y 


Overall average of the i-th attribute 






’ ^iv 1 ) ■ ■ ■ ) 


( 1 ) 


(l) 

where Sy is the set of all tuples with outcome y and Xjy is the value of the i -th attribute of the j-th tuple among all the £y 
tuples with outcome y . The average of the i-th attribute from all tuples within each outcome class is given by: 


Vy G y|xW = 

and the overall average of the i-th attribute from all tuples is : 


(i) _ l^j=l 


_ 2^yeY2^j=i^jy _ l^yeY^^y^y 

e ~ e 


The square of CR lfTTI between the i-th attribute of the dataset and the outcome or the class attribute is given by 


Crf = 


Y' (Ji) _7f;{i)\2 

l^vdY z^i=i ) 


( 2 ) 


(3) 


(4) 


If the value of the i-th attribute of the dataset and value of the outcome are linearly related, then both the Correlation Coefficient 
and the CR will have same value which is equal to the slope of the dependence. 

The CR is able to capture non-linear dependencies in all other cases. 

B. Example for Computation of Correlation Ratio 


TABLE II: Dataset Descriptions 


Blood Pressure(BP) 

Blood Sugar(BS) 

Age-group 

60 

100 

teenager 

75 

120 

teenager 

70 

90 

teenager 

80 

125 

teenager 

65 

90 

teenager 

80 

110 

middle-aged 

75 

105 

middle-aged 

85 

123 

middle-aged 

72 

92 

middle-aged 

90 

130 

old 

80 

109 

old 

120 

130 

old 

100 

132 

old 

95 

127 

old 

85 

119 

old 


The computation of CR can be illustrated using the following example where we have considered a dataset (shown in Table 
E of 15 patients of different age groups - teenager, middle-aged, old and the class attribute(Y) for the dataset is Age-group. 
































The labels of attribute Age-group(Y) is denoted as y where y ^teenager, middle-aged, old. Let the i-th attribute (where i=l) 
in the dataset is BP. The values of the attribute Blood pressure (BP) for these different sets of patients are given as: 

For y = teenager, Steenager = J = 1, • • ’ ,5 

= 60, 75, 70,80,65 (BP values of 5 teenager patients) 

For y = middle - aged, Smiddie-aged = ); J = 1, ’ ’ ’ ,4 

= {80, 75, 85, 72} (BP values of 4 middle-aged patients) 

For y = old. Sold = ); j = 1, • • • ,6 

= {90, 80,120,100, 95, 85} (BP values of 6 old patients) 


For y = teenager, average of BP(i-th attribute), 

(BP) _ jQ 
'^teenager ' ^ 

For y = middle — aged, average of BP(i-th attribute), 

= 78 

middle-aged 

For y = old, average of BP(i-th attribute), 

- 95 

Overall average of BP for these three different age-group patients, = 82 

The weighted sum of square of the differences between the average BP of each group of patients and the overall average is 
= 5(70 - 82)2 _ 32)2 5^95 _ 32)2 

= 1798 

whereas the overall sum of the squares of the differences between the individual BP and the overall average BP is: 


(60 - 82)2 (75 _ 32)2 (79 _ 32)2 (39 _ 32)2 (95 _ 32)2 (39 _ 32)2 (75 _ 32)2 (35 _ 32)2 (72 _ 32)2 (99 _ 

82)2 (39 _ 32)2 (420 _ 82)2 (499 _ 32)2 (95 _ 32)2 (35 _ 32)2 ^ 3449 


Thus, from equation (7), 


= Mi = 0.572 


CrBP = 0.756 


C. Proposed Algorithms 

The CR approach that has been discussed above is basically applicable to quantitative data. So, we propose an approach based 
on the concept of CR which will be applicable on dataset having nominal or categorical attributes. 



Algorithm 1 Constructing CR based Decision Tree(D,A^) 

1 : //Inputs: Dataset D and node Ng 

2: Let A = Ai, A 2 , ■ ■ ■ , An be the set of n attributes for the tuples in D 
3: if all the tuples in D have the same class label then 
4: Return Ni as a leaf node labelled with the class label 

5: else 

6 : for each attribute Ai do 

1: Cn = Correlation_ratio(Ai, y) , where Y is the class attribute 

8 : Insert in set CR, where CR is the set of Correlation ratios. 

9: end for 

10: r = max(CR) 

11 : if multiple Ai have == r then 

12 : choose the attribute Ai as the splitting attribute which has most of the possible distinct values present in D 

13: else 

14: Choose the attribute Ai as the splitting attribute which has == r 

15: end if 

16: Label node Nf, with attribute Ai 

17: Let attribute Ai has m distinct values 

18: Divide D into m partitions D = Di, D 2 , ■ ■ ■, corresponding to each distinct value of Ai respectively, and create a 

child node Nij corresponding to each partition from node N£ with corresponding distinct value of attribute Ai as the 
label on the branch 
19: for each partition Dj in D do 

20 : if that partition is empty then 

21 : Label node Nij as a leaf node with the majority class in D 

22: else 

23: Call DecisionTree(Dj,Afj) 

24: end if 

25: end for 

26: end if 


Algorithm 2 Compute CR(Ai,y) 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 


14: 

15: 


16: 


17: 

18: 


//Inputs: Ai,Y 

Let attribute Ai has m distinct values with respect to D 

Let class attribute Y has I distinct labels Yi,l2,. Yi 

for each class label Yj of Y do 

for each distinct value ^ of Ai with class label Yj do 
fk = frequency{Aj^^) 

Insert in set Fy^ 
end for 


my- = max{Fy) 

_ - rriY- 

xir = , where ty. is the total occurrence 

Y iy, 

end for 

for each attribute Aj do 

Call Awg(Yi,Y2,-;Yl,myymyy...my,tyytyy...,ty) 


of records in the dataset D with class label Yj. 


// This function returns the value x* 




,where 


x* is the overall average of the i-th attribute 

end for 

Call Dism(Yi,Y2,■■■■,Yi,ty^,.Xy , Xy Xy ,xA //which returns the dispersion among individual classes as 

din = '^y-^ytYj{xy- — S*) , where Xy, is the average of the i-th attribute within each outcome class Yj. 

Call \A^OW{Yi,Y2,-;yum,x\iy^),x\2Y)Y-,x\rn,Yi),x\iy^),x\2,Y2)Y-,x\„^y.^),.-,x\lYi)^x\2,Yi), ■■■,x\m,Yi),xA 

// which returns the dispersion across whole population as dov = ) ~ 2;*)^, where x^a.Yj is the 

frequency of occurrence of the a — th distinct value of Ai with class label Yj . 

Compute Correlation Ratio square Cr^Ai as the ratio of din and dov 
Return square-root of Cr^Ai as Correlation Ratio 








We show in Algorithm [T] how to create a Decision Tree using CR as the splitting criterion. A root node is built corresponding 
to the whole dataset. CR based approach is used to split the dataset further. CR between each attribute and the class attribute 
is calculated at each level of Decision Tree construction and the attribute with the highest CR value with the Class attribute 
is chosen as the partitioning attribute for the dataset. The root node is labelled with the corresponding splitting attribute. The 
subtrees of the root node are built using the different distinct values of the splitting attribute as the branch labels and child 
nodes are created from the root node for each splitted sub-dataset respectively. If in any partition, all the tuples have the same 
class label, then label the corresponding leaf node with the corresponding class label. On the other hand, if any partition is 
empty, then mark the corresponding leaf node with the majority class label in its parent’s partition. Iterate the same process 
until for each partition, all the data points have the same class labels. 

In Algorithm 1^ we have shown various steps to compute CR. When calculating the average value of the i-th attribute within 
each outcome class we have taken the ratio of the highest frequency value of occurrence for a distinct value of i-th attribute 
for that class and the total occurrence of records with that outcome class. Thereafter the overall average of the i-th attribute 
is calculated. Then the ratio of the dispersion among individual classes and the dispersion across the whole population for the 
i-th attribute is calculated which is actually the square of the CR for the i-th attribute from which the square root is calculated 
to get the actual CR value. 


The computation of signihcance of an attribute using Algorithm 2 is illustrated using the following example in Table III 


TABLE III: Example dataset 


Class 

Temperature 

Hot 

Mild 

Cool 

No 

2 

1 

0 

Yes 

1 

1 

2 


.^( 1 ) _ 
•^No — 


2 

3 


0.667 


.^.( 1 ) _ 
•^Yes — 


2 

4 


= 0.5 


a;(i) = I = 0.571 




3 * (0.667 - 0.571)2 (-q 5 0 . 571)2 


Temperature _ q ^^^^^2 _ 0.571)2 (q - 0.571)^ -P 

(1 - 0.571)2 _ 0.571)2 (2 - 0.571)2 

= 0.00963 

^'^Temperature ~ 0.098 


In Table III we consider Temperature as the first attribute of a dataset and it has three possible values - Hot, Mild, Cool. 
There are two classes - No and Yes. The frequencies of the attribute values for each of the class is shown using the numeric 
values in each cell of the table. The maximum frequency value for the attribute Temperature is used to calculate the average 
weight of the attribute in each class. The overall average weight of the attribute is the ratio of the summation of the 
maximum frequencies of the two classes and the total number of instances in the two classes. The signihcance of the attribute 
Temperature for predicting class Y, Crxemperature is calculated as shown in the example. The continuous attributes in the 
datasets taken from UCI machine learning repository QSl were discretized. In each level while constructing the Decision Tree, 
we have considered the attribute which has the highest value of CR with the class attribute. 


IV. Observations on Benchmark Healthcare Datasets 

Several datasets like Pima Indian Diabetes dataset. Liver Disorder dataset. Mammography Masses dataset. Breast Cancer 
dataset. Hepatitis dataset. Post-operative dataset, ILPD dataset and Spect-heart datasets from UCI machine learning repository 
were considered for performance evaluation of our proposed approach. 


A. About the datasets 

Table |IV] shows the characteristics of the datasets considered here. 












TABLE IV: Nature of the Datasets 


Dataset 

Number of 
instances 

No. of attributes 

Characteristics of attributes 

Pima Indian Diabetes 

768 

9 

All have same no. of distinct values 

Mammography 

961 

6 

Different no. of distinct values 

Breast Cancer 

699 

10 

All have same no. of distinct values 

Hepatitis 

155 

20 

Different no. of distinct values 

Post-operative 

90 

9 

Different no. of distinct values 

ILPD 

583 

10 

Different no. of distinct values 

Spect-heart 

267 

23 

All have same no. of distinct values 

Statlog(heart) 

270 

13 

Different no. of distinct values 


The Pima Indian Diabetes dataset has overall 9 attributes including the class attribute which is categorical {tested positive for 
diabetes{\), tested negative for diabetes{0)). All the attributes are numeric-valued except the class attribute. There were 768 
instances. Among these 500 instances are having class value 0 and 268 instances are having class value 1. 

The Mammography Masses dataset consists of 961 instances and 6 attributes in which one of the attribute is the class attribute 
(possible values - 0 and 1). One attribute was of type integer and the other attributes are nominal or ordinal. The integer 
attribute was discretized into 5 different values. Two of the ordinal attributes have 5 different values and remaining two ordinal 
attributes have 4 different values. There were some missing values in almost all of the attributes. Class distribution was like : 
benign(O)- 516, malignant(l)- 445. 

The Breast Cancer dataset has 699 instances and total 10 attributes. The class attribute takes two possibles values - 2 for benign 
and 4 for malignant. Except the class attribute and another attribute which is indicating the id number, rest of the attributes 
can take values in the the range 1-10. There are 458 instances which are Benign and 248 Malignant instances. Missing values 
were present in 16 instances. 

The Hepatitis dataset has overall 155 instances and 20 attributes of different types like categorical, integer and real. Six of 
the attributes are of integer type and were discretized into hve discrete values. Thirteen attributes are categorical each having 
two possible values. The class attribute is categorical and can take two possible values - DIE(32 instances) and LIVE(123 
instances). There are missing values in many attributes. 

The Post-operative dataset is having 9 attributes (including the class attribute) and 90 instances. Attributes are of different types 
like - categorical and integer. There are seven categorical attributes and one integer attribute. The class attribute is categorical 
and can take three possible values - I (patient sent to Intensive Care Unit): 2 instances, S (patient prepared to go home): 24 
instances and A (patient sent to general hospital floor): 64 instances. There were no missing values. 

The Indian Liver Patient Dataset(ILPD) contains 583 instances and 10 attributes of different categories like integer and real. 
Among 10 attributes 9 are integer type and 1 attribute is categorical. Each of the integer type attributes are discretised into 
hve distinct categorical values. The class label attribute can take two possible values ’1’ and ’2’. There are 416 instances with 
class label ’1’ and ’167’ instances with class label ’2’. 


The Spect Heart dataset consists of 267 instances and 23 attributes including the class attribute. All the attributes are binary. 
The class attribute can have 2 possible values ’0’ and ’1’ and class distribution is - 55 examples with class label ’0’ and 212 
examples with class labels ’1’. It has no missing values. 

The Statlog (heart) dataset has 13 attributes where one of the attributes is the class attribute. The attributes are of different 
types like Real, Ordered, Nominal and Binary and there are total 270 observations and two classes. Out of six real attributes, 
hve are transformed into hve categorical values and the remaining one real attribute is transformed into four categorical values. 
Three attributes are binary. One attribute is ordered and has three different values. Three attributes are nominal out of which 
one attribute is having four different values and two attributes is having three different values. 


The continuous attributes of different datasets are discretized. The natures of the datasets after discretization has been shown 


in Table IV For most of the datasets k-fold cross validation has been used where the dataset is divided into k disjoint subsets 
and (k-1) subsets are used for training and the remaining subset is used for testing. This process is repeated k-number of times 
and the results of all the iterations are combined togetherjO. For the Post-Operative dataset, it is divided into training and test 
sets in the ratio of 70:30. In this case, as the dataset is very small, so cross validation is not used. The Spect-heart dataset is 
already divided into separate training (80 instances) and test sets(187 instances). 















B. Result and Analysis 


Next we have applied our proposed technique on the datasets discussed in Subsection IV-A 


TABLE V; Experimental Results 


Dataset 

Cross validation 

IG Accuracy 

CR Accuracy 

Pima Indian 
Diabetes 

5-fold 

70.93% 

69.24% 

Mammography 

5-fold 

94.11% 

93.88% 

Breast Cancer 

5-fold 

91.03% 

90.03% 

Hepatitis 

2-fold 

71.19% 

73.78% 

Post-operative 

70 : 30 

62.96% 

62.96% 

ILPD 

5-fold 

66.89% 

67.57% 

Spect-heart 

80 training and 

187 test 

instances 

74.33% 

74.33% 

Statlog(heart) 

2-fold 

74.07% 

74.4% 


Table [V] shows that for Pima Indian Diabetes, Mammography and Breast Cancer datasets the IG based approach has performed 
slightly better than CR based approach. Eor two of these datasets (Pima Indian Diabetes, Breast Cancer) more number of 
attributes are there and all of the attributes have same number of distinct values. One dataset (Mammography) has different 
numbers of distinct values for the attributes but less number of attributes. 

Eor the Spect-heart dataset both IG and CR approaches have given same performance (74.33% accuracy). This dataset also has 
same number of distinct values for all the attributes and has lots of attributes compared to the number of instances. Eor the 
Post-operative patient dataset both the approaches have given 62.96% accuracy. The reason behind the not-so-good performance 
can be attributed to the fact that there were less number of instances (90), more number of attributes (9) and more number 
of classes (3) compared to the other datasets. Also this dataset has different number of distinct values for different types of 
attributes. 

Eor the ILPD dataset, almost all the attributes have same number of distinct values except one attribute. The CR based approach 
performed slightly better than the IG based approach. The CR approach outperformed the IG based approach for the Hepatitis 
dataset. 6 attributes have same number of distinct values and rest 13 attributes have less number of distinct values. The CR 
approach has given slightly better result for the Statlog (heart) dataset where the attributes have different numbers of distinct 
values and there are total 270 observations and two classes. 

It is therefore observed from the analysis of the result that generally our proposed approach can handle datasets with same 
as well as different numbers of distinct valued attributes almost equally well because it is not biased towards attributes with 
large number of distinct values. On the other hand IG approach prefers attributes with many distinct values. In healthcare 
datasets where there are large number of attributes and different attributes have different numbers of distinct values, the IG 
based approach prefers attributes with more number of distinct values rather than attributes which have less number of distinct 
values even if those attributes may be more significant for classification. This gives a reason for lesser performance of the 
IG based than our proposed approach in those cases. Also another observation that has been made out of the results is that 
for smaller datasets where less number of instances and comparatively more number of attributes are there like Hepatitis and 
Statlog (heart)datasets, the proposed approach performs slightly better than the IG based approach. Eor this to be taken as a 
proven fact some more datasets need to be analysed in future. 

V. Conclusion 

Healthcare datasets are available in plenty. The nature or the distribution patterns of such healthcare datasets may also vary. 
However we observe that such datasets generally have large number number of attributes and different attributes have different 
numbers of distinct values. Thus, applying existing IG based splitting criterion may not give good accuracy for all cases. So, 
in this paper, CR based Decision Tree learner is proposed. This technique serves as a complement of IG based technique i.e., 
when IG fails, CR based technique succeeds. We demonstrated this fact using some benchmark healthcare datasets. In future 
we would like to explore some more such datasets and apply our proposed technique. 
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